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@ A control device for an image input apparatus 
which is equipped with an optical system having a 
magnification varying lens, includes a monitor for 
displaying input images, an input device which en- 
ables an arbitrary position on a display screen of the 
monitor to be designated, a calculation device for 
calculating the distance between a predetermined 
position on the display screen and the arbitrary 
position on the basis of zooming information of the 
optical system, and a controller for controlling the 
image input apparatus in accordance with the cal- 
culation results obtained by the calculation device. 
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Background of the Invention: 

Field of the Invention: 

This invention relates to a control device for an 
image input apparatus and, more specifically, to a 
control device which is suitable for remotely op- 
erating a camera as in the case of a video con- 
ference system. 

Description of the Related Art: 

With the improvement of computers in terms of 
image processing capability, there have been pro- 
posed various techniques in which camera oper- 
ations, such as zooming, panning and tilting, are 
performed by operating a computer while the pho- 
tographic image is being displayed on a monitor 
screen of the computer. In particular, in a video 
conference system, it is desirable that the orienta- 
tion (pan, tilt), magnification, etc., of the camera at 
the other end of the communications line be re- 
motely controllable. For that purpose, there has 
been proposed, for example, a system in which 
camera operation factors, which are to be con- 
trolled by using a mouse or the like, are displayed 
on a part of the monitor screen. 

However, in the above system, it is rather 
difficult to perform fine adjustment. Moreover, it is 
by no means easy to determine which factor Is to 
be controlled, and to what degree, so that the 
operator has to depend on trial-and-error methods. 
When a camera is to be remotely controlled as in 
the case of a video conference system, the time 
lag involved in transmitting the control signal must 
also be taken into account. In addition, the image is 
subjected to high-efficiency coding before being 
transmitted. Thus, the image quality is generally 
rather poor when the image information changes 
fast as in the case of panning, thereby making the 
camera operation still more difficult. In other words, 
a fine adjustment is impossible. 

Summary of the Invention: 

It is an object of the present invention to pro- 
vide an image-input-apparatus control device im- 
proved in operability. 

In accordance with an embodiment of the 
present invention, there is provided a control de- 
vice for an image input apparatus which is 
equipped with an optical system having a mag- 
nification varying lens, the control device compris- 
ing display means for displaying input images, 
input means which enables an arbitrary position on 
a display screen of the display means to be des- 
ignated; calculation means for calculating the dis- 
tance between a predetermined position on the 



display screen and the arbitrary position on the 
basis of zooming information of the optical system, 
and control means for controlling the image input 
apparatus in accordance with calculation results 
5 obtained by the calculation means. 

Other objects and features of the present in- 
vention will become apparent from the following 
detailed description of the invention and the ac- 
companying drawings. 

10 

Brief Description of the Drawings: 

Fig. 1 is a schematic block diagram showing an 
embodiment of the present invention; 
r6 Rg. 2 is a schematic diagram of the embodi- 
ment connected to a communications network; 
Rg. 3 is a flowchart illustrating a first operation 
of this embodiment; 

Rg. 4 is a diagram illustrating how distances as 
20 displayed on a monitor screen are related to the 
corresponding angles of view; 
Rg. 5 is a flowchart illustrating a second opera- 
tion of this embodiment 

Rg. 6 is a flowchart illustrating a third operation 
25 of this embodiment; and 

Rg. 7 is a flowchart illustrating a fourth operation 
of this embodiment. 

Detailed Description of the Preferred Embodiment: 

30 

An embodiment of the present invention will 
now be described with reference to the drawings. 

Rg. 1 is a schematic block diagram showing 
an embodiment of the present invention as applied 

35 to a terminal in a video conference system. Nu- 
meral 10 indicates a camera for photographing a 
user of the system; numeral 12 indicates a photog- 
raphing zoom lens unit; numeral 14 indicates a 
zoom control circuit for moving a zooming lens 12a 

40 of the lens unit 12 in the direction of the optical 
axis; numeral 16 indicates a focus control circuit for 
moving a focusing lens 12b of the lens unit 12 in 
the direction of the optical axis; numeral 20 in- 
dicates an image sensor which converts optical 

45 images obtained by the lens unit 12 and an ap- 
erture 16 to electric signals; and numeral 22 in- 
dicates a camera signal processing circuit for con- 
verting the electric signals obtained by the image 
sensor 20 to video signals. 

so Numeral 24 indicates a pan control circuit 

which is on a pan head 23 and which moves the 
photographic optical axis of the camera 10 to the 
right and left; numeral 26 indicates a tilt control 
circuit which is on the panhead 23 and which 

55 moves the photographic optical axis of the camera 
10 up and down; and numeral 28 Indicates a cam- 
era control circuit for controlling the camera 10 as 
a whole. 



3 



EP0 637 168 A1 



4 



Numeral 30 indicates a computer constructed 
in the same way as ordinary computers; numeral 
32 indicates a CPU for overall control; numeral 34 
indicates a ROM; numeral 36 indicates a RAM; 
numeral 38 indicates a video Interface to which 
output video signals from the camera 10 are input; 
and numeral 40 indicates a communications inter- 
face which transmits and receives data and control 
signals to and from an external communications 
network and transmits and receives control signals 
to and from the camera 10. 

Numeral 42 indicates a coordinate input device 
consisting of a digitizer, a mouse or the like; nu- 
meral 44 indicates an interface for the coordinate 
input device 42; numeral 46 indicates a video 
memory (VRAM); and numeral 48 indicates a dis- 
play control device for controlling the image dis- 
play of a monitor 50 consisting of a CRT, a liquid 
crystal display or the like. 

As shown in Fig. 2, a number of terminals as 
shown in Fig. 1 are connected to each other 
through the intermediation of a communications 
network. 

Next, the operation of this embodiment will be 
described with reference to Figs. 1 and 3. This 
embodiment functions in a particularly effective 
manner when applied to a case where it is used to 
remotely control a camera at an image transmis- 
sion end terminal from an image reception end 
terminal in a video conference which is being ex- 
ecuted between terminals. However, for conve- 
nience of description, the operation of this embodi- 
ment will be explained with reference to a case 
where the camera 10 is operated within the system 
shown in Fig. 1. The features of this embodiment 
as utilized in the above two cases are the same 
except for the fact that the processes of image 
coding and decoding are omitted in the latter case. 

A photographic image obtained by the camera 
10 is written to the memory 46 through the video 
interface 38. The display control circuit 48 succes- 
sively reads image data stored in the video mem- 
ory 46, whereby the monitor 50 is controlled to 
display the image. 

The user designates an arbitrary position (x, y), 
which he or she intends to be the center, through 
the coordinate input device 42 (S1). The CPU 32 
calculates the disparity (AX, AY) between the des- 
ignated position (x, y) and the coordinates (a, b) of 
the center of the photographic image displayed on 
the screen (when no window display system is 
adopted, it is the center of the screen of the 
monitor 50 and, when a window display system is 
adopted, It is the center of the display window of 
the photographic image) (S2). That is, the CPU 32 
calculates the following values: 

AX = x - a 



AY = y - b 

Then, the CPU 32 transfers a movement command, 
5 e.g., a command Mov (AX, AY), to effect move- 
ment through a distance corresponding to the dis- 
parity (AX, AY) to the camera control circuit 28 of 
the camera 10 through the communications inter- 
face 40 (S3). 

to Upon receiving this movement command (S4), 

the camera control circuit 28 first obtains zooming 
position information of the zooming lens 12a from 
the zoom control circuit 14 (S5), and determines 
the conversion factor k of the amount of movement 

15 from the zooming position information thus ob- 
tained (S6). That is, the photographic image is 
displayed in a size corresponding to the variable 
magnification of the lens unit 12. For example, as 
shown in Fig. 4, distances which appear the same 

20 when displayed on the screen of the monitor 50 
are different from each other in the actual field 
depending upon the magnification, i.e;, the angle of 
view. In view of this, it is necessary to convert a 
distance on the monitor screen to an amount of 

25 movement corresponding to the angle of view (the 
pan angle and the tilt angle). For this purpose, the 
camera control circuit 28 is equipped with a con- 
version table for determining the conversion factor 
k. 

so The determined conversion factor k Is multi- 

plied by the parameters AX and AY of the move- 
ment command Mov (AX, AY) from the computer 
30 to calculate the actual amount of movement 
(AXr, AYr) (S7). That is, the following values are 

35 calculated: 

AXr = kAX 
AYr = kAY 

40 

The camera control circuit 28 determines the pan 
and tilt angles of rotation in accordance with the 
actual amount of movement (AXr, AYr) (S8) to 
control the pan control circuit 24 and the tilt control 
45 circuit 26, thereby pointing the photographic optical 
axis of the camera 10 in the designated direction 
(S9). 

While In Fig. 3 the camera control circuit 28 of 
the camera 10 calculates an amount of movement 

so of the camera 10 with respect to the amount of 
movement as designated on the monitor screen, it 
is naturally also possible for this calculation to be 
performed by the CPU 32 of the computer 30. Fig. 
5 shows a flowchart for the latter case which differs 

55 from the above-described case in that the zooming 
position information of the zooming lens 12a is 
transferred from the camera 10 to the computer 30, 
which calculates the pan and tilt angles of move- 
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ment and transfers them to the camera 10. 

That is, the user designates an arbitrary posi- 
tion (x, y), which he or she intends to be the center 
of the photographic image displayed on the moni- 
tor 50, by using the coordinate input device 42 s 
(S11). The CPU 32 calculates the disparity (AX. 
AY) between the designated position (x, y) and the 
coordinates (a, b) of the center of the photographic 
image as displayed on the screen (when no win- 
dow display system is adopted, it is the center of io 
the screen of the monitor 50 and, when a window 
display system is adopted, it is the center of the 
display window of the photographic image) (S12). 
Then, the CPU 32 requires the camera 10 to pro- 
vide zooming position information (SI 3). 75 

Upon the request of zooming position informa- 
tion from the computer 30, the camera control 
circuit 28 of the camera 10 obtains zooming posi- 
tion information from the zoom control circuit 14 
(S14), and transfers it to the computer 30 (S15). 20 

The CPU 32 determines the conversion factor 
k from the zooming position information from the 
camera 10 (S16, S17). In this example, the CPU 32 
is equipped with a conversion factor table for con- 
verting a distance on the screen of the monitor 50 25 
to an amount of movement corresponding to the 
angles of view (the pan and tilt angles), and deter- 
mines the conversion factor k. 

The CPU 32 multiplies the determined conver- 
sion factor k by the previously calculated AX and so 
AY to calculate the actual amount of movement 
(AXr, AYr) (S18), and determines the pan and tilt 
angles of rotation corresponding to the calculated 
actual amount of movement (AXr, AYr). transmit- 
ting a movement command of that angle of rotation 35 
to the camera 10 (S20). 

The camera control circuit 28 of the camera 1 0 
receives the movement command from the com* 
puter 30 (S21), and controls the pan control circuit 
24 and the tilt control circuit 26 in accordance with 40 
the command to point the photographic optical axis 
of the camera 10 in the designated direction (S22). 

Next, a case in which a photographic range is 
designated by two points on the screen of the 
monitor 50 will be described with reference to Fig. 45 
6. The user designates two points (x1, yl) and <x2, 
y2) in the photographic image plane, which is dis- 
played on the monitor 50, by the coordinate input 
device 42 (S31). The CPU 32 calculates a middle 
point (xO, yO) thereof from the designated two so 
points (x1, y1) and (x2, y2) (S32). The CPU32 
calculates the difference (AX, AY) between the 
middle point <xO, yO) and the coordinates (a. b) of 
the center of the photographic-image displaying 
portion of the monitor 50 (when no window display 55 
system is adopted, It is the center of the entire 
screen of the monitor 50, and when a window 
display system is adopted, it is the center of the 



photographic image display window) (S33). That is, 
the CPU 32 calculates the following values: 

AX = xO - a 

AY » yO - b 

Further, the CPU calculates the difference (Ax, 
Ay) between the two designated points, that is. the 
following values (S34): 

Ax = xl - x2 

Ay = y1 - y2 

The CPU 32 transfers a movement command 
in which the differences (AX, AY) and (Ax, Ay) are 
used as parameters to the camera 10 (S35). The 
camera control circuit 28 of the camera 10 receives 
this command thus transferred (S36). and obtains 
zooming position information of the zooming lens 
12a from the zoom control circuit 14 (S37), deter- 
mining the conversion factor k of the pan and tilt 
amounts of movement from the zooming position 
information thus obtained (S38). 

The camera control circuit 28 multiplies the 
determined conversion factor k by the parameters 
AX and AY of the movement command from the 
computer 30 to calculate the actual amount of 
movement (AXr, AYr), that is, the following values 
(S39): 

AXr - kAX 
AYr = kAY 

Further, the camera control circuit 28 calculates the 
amount of movement to a desired zooming posi- 
tion, Az, from the parameters Ax and Ay form the 
compute 30 and the factor k (S40). 

The camera control circuit 28 calculates the 
actual amounts of movement corresponding to the 
actual amount of movement (AXr, AYr) calculated 
in step S39, and calculates the amount of zooming 
movement corresponding to the amount of move- 
ment Az calculated in step 340 (S41). In then 
controls the pan control circuit 24, the tilt control 
circuit 26 and the zoom control circuit 14, pointing 
the photographic optical axis of the camera 10 in 
the designated direction and changing the mag- 
nification of the lens unit 12 (S42). 

The above-described embodiment is not effec- 
tive when the user wishes to observe ranges other 
than that displayed on the monitor screen. In such 
a case, the user has to move the zooming lens 12a 
of the camera 10 to wide-angle end through an- 
other operation, and then perform the above opera- 
tion. These procedures could be simplified by the 
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As can be readily understood from the above 
description, in accordance with this embodiment, a 
visual and intuitive camera operation can be re- 
alized, thereby attaining an improvement in oper- 
ablllty. In particular, this embodiment proves mar- 
kedly effective in cases where remote control Is to 
be performed. 

While the above embodiment has been de* 
scribed as applied to optical zooming, which is 
effected by moving a zooming lens, it is also 
applicable to electronic zooming, in which captured 
image data is electronically processed. 

A control device for an image input apparatus 
which is equipped with an optical system having a 
magnification varying lens, includes a monitor for 
displaying input images, an input device which 
enables an arbitrary position on a display screen of 
the monitor to be designated, a calculation device 
for calculating the distance between a predeter- 
mined position on the display screen and the ar- 
bitrary position on the basis of zooming information 
of the optical system, and a controller for control- 
ling the image input apparatus in accordance with 
the calculation results obtained by the calculation 
device. 

Claims 



following procedures: the CPU sets imaginary 
screens above and below, and on the right and left- 
hand sides, of the display screen displaying an 
image that is being photographed. These imaginary 
screens may be adjacent to, or partly overlap, or s 
spaced apart from, the actual display screen. Fig. 7 
shows the flowchart of an operation utilizing such 
imaginary screens. 

The user selects one of the imaginary screens 
by the coordinate input device 42 or the like (S51). to 
The CPU 32 calculates the disparity between the 
center (at, b1) of the selected imaginary screen 
and the coordinates (a, b) of the center of the 
displayed photographic image (when no window 
display system is adopted, it is the center of the rs 
screen of the monitor 50 and, when a window 
display system is adopted, It Is the center of the 
display window of the photographic image), that is, 
the CPU calculates the following values (S52): 

20 

AX = al - a 
AY = b1 - b 

Then, the CPU 32 transfers a movement command as 
corresponding to the difference (AX, AY), e.g., Mov 
(AX, AY), to the camera control circuit 28 of the 
camera 10 through the communications interface 
40 (S53). 

The camera control circuit 28, which receives so 
this movement command (S54), first obtains the 
zooming position information of the zooming lens 
12a from the zoom control circuit 14 (S55), and 
then determines the conversion factor k of the 
amount of movement from the obtained zooming ss 
position information, as In the abvoe-described 
case (S56). 

The camera control circuit 28 multiplies the 
determined conversion factor k by the parameters 
AX and AY of the movement command Mov (AX, <o 
AY) to calculate the actual amount of movement 
(AXr, AYr) (S57). That Is, 

AXr = kAX 

45 

AYr = kAY 



Further, the camera control circuit 28 determines 
the pan and tilt angles of rotation corresponding to 
the actual amount of movement (AXr, AYr), (S58), 
and controls the pan control circuit 24 and the tilt 
control circuit 26 to point the photographic optical 
axis of the camera 10 in the designated direction. 

As stated above, the present invention is obvi- 
ously also applicable to a case where a camera at 
an image-transmission-end terminal Is to be re- 
motely controlled from an image-reception-end ter- 
minal in a video conference or the like. 



1. A control device for an image input apparatus 
which Is equipped with an optical system hav- 
ing a magnification varying lens, said control 
device comprising: 

(a) display means for displaying input im- 
ages; 

(b) input means which enables an arbitrary 
position on a display screen of said display 
means to be designated; 

(c) calculation means for calculating the dis- 
tance between a predetermined position on 
said display screen and said arbitrary posi- 
tion on the basts of zooming information of 
said optical system; and 

(d) control means for controlling said image 
input apparatus in accordance with calcula- 
tion results obtained by said calculation 
means. 

2. A control device according to claim 1 , wherein 
said control means causes the optical axis of 

so said optical system to move in such a way that 

said arbitrary position on said display screen 
comes to coincide with said predetermined 
position. 

55 3. A control device according to claim 2, wherein 
said predetermined position on said display 
screen is the center of said display screen. 



5 



9 



EPO 637 168 A1 



10 



4. An image input apparatus comprising: 

(a) zoom means for zooming input images; 

(b) display means for displaying input im- 
ages; 

(c) input means which enables an arbitrary 
position on a display screen of said display 
means to be designated; 

(d) calculation means for calculating the dis- 
tance between a predetermined position on 
said display screen and said arbitrary posi- 
tion on the basis of zooming Information of 
said optical system; and 

(e) control means for controlling said image 
input apparatus in accordance with calcula- 
tion results obtained by said calculation 
means. 

5. An image input apparatus according to claim 4, 
further comprising an optical system having a 
magnification varying lens, said zoom means 
causing said magnification varying lens to 
move in the direction of an optical axis of said 
optical system. 

6. An image input apparatus according to claim 5, 
wherein said control means causes the optical 
axis of said optical system to move in such a 
way that said arbitrary position on said display 
screen comes to coincide with said predeter- 
mined position. 

7. An image input apparatus according to claim 6, 
wherein said predetermined position on said 
display screen is the center of said display 
screen. 

a. An image input apparatus according to claim 6, 
further comprising a panhead for moving the 
optical axis of said optical system. 

9. An image input apparatus according to claim 4, 
wherein said input means enables a plurality of 
arbitrary positions to be designated. 

10. An image input apparatus according to claim 9, 
wherein said calculation means calculates the 
distance between a middle position of said 
plurality of arbitrary positions and said pre- 
determined position. 

11. An image input apparatus according to claim 
10, further comprising an optical system hav- 
ing a magnification varying tens, said zoom 
means causing said magnification varying lens 
to move in the direction of an optical axis of 
said optical system. 



12. An image input apparatus according to claim 
11, wherein said control means causes the 
optical axis of said optical system to move in 
such a way that said middle position of said 

5 plurality of arbitrary positions on said display 

screen comes to coincide with said predeter- 
mined position. 

13. An image input apparatus according to claim 
70 12, wherein said predetermined position on 

said display screen is the center of said dis- 
play screen. 

14. An image input apparatus comprising: 

76 (a) an optical system which inputs images; 

(b) display means for displaying input im- 
ages; 

(c) input means which enables an arbitrary 
position on a display screen of said display 

20 means to be designated; 

(d) calculation means for calculating the dis- 
tance between the center of said display 
screen and said arbitrary position; and 

(e) control means which causes a displayed 
25 image to move in such a way that said 

arbitrary position on said display screen 
comes to coincide with the center of said 
display screen. 

30 15. An image input apparatus according to claim 
14, wherein said optical system has a mag- 
nification varying lens. 

16. An image input apparatus according to claim 
ss 15, wherein said calculation means calculates 

said distance in accordance with an angle of 
view of said optical system. 

17. An image input apparatus according to claim 
40 12. wherein said control means causes an op- 
tical axis of said optical system to move. 
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AX=xO-a. AY=yo-b 



T 



-S32 



OBTAIN DISPARITY ( d x. Ay> 
BETWEEN TWO POINTS 
(x1.y1).(x2.y2) 



X 



S33 



S34 



TRANSMIT UX.AY>.Ux.4y) 
THROUGH MOVEMENT COMMAND 



S35 



S37 



RECEIVE ( aX. aY ), (ax, Ay I 
THROUGH MOVEMENT COMMAND 



S3B- 



T 



OBTAIN Z00MIN6 POSITION 
INFORMATION FROM ZOOM 
CONTROL CIRCUIT 



S39- 



S40< 



I 



DETERMINE CONVERSION FACTOR k 
FROM CONVERSION TABLE 



T 



CORRECT aX. aY WITH FACTOR k 
AXr = kAX 
AYr = kAY 



S41 



T 



CALCULATE AMOUNT OF MOVEMENT 
aZ TO DESIRED ZOOMING POSITION 
FROM ax. Ay AND FACTOR k 



S42 



I 



DETERMINE PAN. TILT AND ZOOM 
AMOUNTS OF MOVEMENT IN 
ACCORDANCE WITH a Xr.AYr.AZ 



X 



DRIVE MOTOR BY DETERMINED 
AMOUNTS OF MOVEMENT 

" I ■ " 



12 



EPO 637 168 A1 



COMPUTER 30 

C START ) 
t 



F I G.T 



SELECT ONE OF UPPER. LOWER. 
RIGHT AND LEFT ADJACENT 
SCREENS 



I 



CALCULATE DISPARITY UX. AY ) 

BETWEEN CENTER (at, bl) 

OF SELECTED IMAGINARY SCREEN 

AND CENTER (a, b) 

OF MONITOR SCREEN 

4X = ai-a.4Y=bi-b 



I 



S52 



c 



COMMUNI- 
CATION 



TRANSMIT MOVEMENT COMMAND 
MovUXmY) 



-S53 



S54 



S55 



SS6 



S57 



S58- 



S59- 



CAMERA 10 



RECEIVE MOVEMENT COMMAND 
Mov(aX.4Y) 



T 



OBTAIN ZOOMING POSITION 
INFORMATION FROM ZOOM 
CONTROL CIRCUIT 



T 



DETERMINE CONVERSION FACTOR k 
FROM CONVERSION TABLE 



i 



CORRECT AHLAi WITH FACTOR k 
AXr = k4X 
dYi = k4Y 



X 



DETERMINE PAN AND TILT 
AMOUNTS OF MOVEMENT IN 
ACCORDANCE WITH UXr.^Yr) 



I 



ORIVE MOTOR BY DETERMINED 
AMOUNTS OF MOVEMENT 



C END ) 
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